Method, system and device for security inspection

ABSTRACT

The present disclosure relates to a method, a system and a device for security inspection, pertaining to the field of security inspection. The method includes: before a baggage enters a security inspection machine and/or after the baggage leaves the security inspection machine, acquiring information about the baggage and information about a subject person; while the baggage is inside the security inspection machine to be scanned, acquiring a scanned image of the baggage; and correlating the information about the baggage, the scanned image of the baggage and the information about the subject person in a storage system, wherein acquiring information about the baggage and information about a subject person includes analyzing a video.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is based upon and claims priority to ChinesePatent Application No. CN 201511020939.5 filed Dec. 29, 2015, the entirecontents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure generally relates to the field of securityinspection, and more particularly, to a method, a system and a devicefor security inspection of a correlated human body and baggage.

BACKGROUND

At present, for security inspection of public on roads, railway stationsand other public places, a security officer generally verifies anidentity of a subject person by checking his or her ID card. The subjectperson's baggage is generally inspected through an image generated byscanning with radiation rays (such as X rays) emitted from a particulardevice (such as a security inspection machine). In addition, amonitoring camera can capture and store videos of an inspection regionin that place.

In the method above, inspection of the identity of the person, thebaggage and the monitoring video of the inspection region are performedindependently from one another without correlation. After a securityaccident happens, it is difficult to track other information only basedon single inspection result (such as a package left in the place or aface of a suspect in the monitoring video).

Therefore, there is a demand for a novel method and a system forsecurity inspection.

The above contents disclosed in the BACKGROUND section are merely forbetter understanding of the background of the present disclosure, itdoes not constitute prior art known to the ordinary skilled in the art.

SUMMARY

In view of the above, the present disclosure provides a method, a systemand a device for security inspection which can provide correlation amonginformation acquired in the security inspection and improve theusability of information acquired in the security inspection.

Additional aspects and advantages of the present disclosure will be setforth in the following description and become apparent from thedescription, or can be partly learned from practice of the presentdisclosure.

According to one aspect of the present disclosure, a method for securityinspection. The method includes: before a baggage enters a securityinspection machine and/or after the baggage leaves the securityinspection machine, acquiring information about the baggage andinformation about a subject person; while the baggage is inside thesecurity inspection machine to be scanned, acquiring a scanned image ofthe baggage; and correlating the information about the baggage, thescanned image of the baggage and the information about the subjectperson in a storage system, wherein acquiring information about thebaggage and information about a subject person includes analyzing avideo.

According to an embodiment of the present disclosure, acquiringinformation about the baggage includes: acquiring a video of thebaggage; and analyzing the video of the baggage to acquire entrancebaggage information about the baggage.

According to an embodiment of the present disclosure, acquiringinformation about a subject person includes: acquiring a baggageretrieving video; and analyzing the baggage retrieving video to acquirethe information about the subject person who retrieves the baggage.

According to an embodiment of the present disclosure, analyzing thebaggage retrieving video to acquire the information about the subjectperson includes: analyzing the baggage retrieving video to acquire exitbaggage information and the information about the subject person; andmatching the exit baggage information with the entrance baggageinformation to identify the baggage when the baggage leaves the securityinspection machine.

According to an embodiment of the present disclosure, analyzing thebaggage retrieving video to acquire the information about the subjectperson includes: matching a time when the baggage enters the securityinspection machine with a time when the baggage leaves the securityinspection machine, to identify the baggage when the baggage leaves thesecurity machine; and analyzing the baggage retrieving video to acquirethe information about the subject person who retrieves the baggage.

According to an embodiment of the present disclosure, acquiring ascanned image of the baggage includes: matching a calculated time whenthe scanned image of the baggage is generated with an actual time whenthe scanned image is generated, to acquire the scanned image of thebaggage.

According to an embodiment of the present disclosure, analyzing a videoincludes: decoding a stream of a video to extract a single frame;marking an interested region and an interested line on an image of theframe; detecting a dynamic object in the interested region; acquiring aforeground view and performing opening and closing process on theforeground view; once in the processed foreground view, the dynamicobject has crossed the interested line, comparing the current frame witha state of the preceding frame and determining whether the current frameis a key frame; if the current frame is a key frame, extracting andstoring information about a baggage of the dynamic object in the keyframe; and if it is not a key frame, extracting a next frame.

According to an embodiment of the present disclosure, analyzing a videoincludes: dividing a real-time stream of video data into data of aplurality of time segments; and processing the data of a plurality oftime segments in parallel.

According to an embodiment of the present disclosure, the method furtherincludes storing the information about the baggage, the scanned image ofthe baggage and the information about the subject person who retrievesthe baggage in a system server.

According to an embodiment of the present disclosure, the informationabout the baggage includes at least one of: an image, a color and asize.

According to an embodiment of the present disclosure, the informationabout the subject person who retrieves the baggage can include at leastone of: an image of the face, a fragment of the baggage retrievingvideo.

According to an embodiment of the present disclosure, the scanned imagecan be an X-ray image.

According to another aspect of the present disclosure, a system forsecurity inspection. The system includes: a baggage-informationacquiring module configured to, before a baggage enters a securityinspection machine and/or after the baggage leaves the securityinspection machine, acquire information about the baggage; abaggage-scanned-image acquiring module configured to, while the baggageis inside the security inspection machine to be scanned, acquire ascanned image of the baggage; a person-information acquiring moduleconfigured to, before a baggage enters a security inspection machineand/or after the baggage leaves the security inspection machine, acquireinformation about a subject person corresponding to the baggage; acorrelation module configured to correlate the information about thebaggage, the scanned image of the baggage and the information about thesubject person in a storage system; and a video analyzing moduleconfigured to analyze a video for the baggage-information acquiringmodule and the person-information acquiring module.

According to an embodiment of the present disclosure, thebaggage-information acquiring module includes: a baggage-video acquiringunit configured to acquire a video of the baggage; a baggage analyzingunit configured to analyze the video of the baggage to acquire entrancebaggage information about the baggage.

According to an embodiment of the present disclosure, theperson-information acquiring module includes: a baggage-retrieving-videoacquiring unit configured to acquire a baggage retrieving video; and abaggage-retrieving analyzing unit configured to analyze the baggageretrieving video to acquire the information about the subject person whoretrieves the baggage.

According to an embodiment of the present disclosure, thebaggage-retrieving analyzing unit includes: a baggage analyzing sub-unitconfigured to analyze the baggage retrieving video to acquire exitbaggage information; a retriever analyzing sub-unit configured toanalyze the baggage retrieving video to acquire the information aboutthe subject person who retrieves the baggage; and a baggage matchingsub-unit configured to match the exit baggage information with theentrance baggage information to identify the baggage when the baggageleaves the security inspection machine.

According to an embodiment of the present disclosure, thebaggage-retrieving analyzing unit includes: a first time-matchingsub-unit configured to match a time when the baggage enters the securityinspection machine with a time when the baggage leaves the securityinspection machine, to identify the baggage when the baggage leaves thesecurity machine; and a retriever analyzing sub-unit configured toanalyze the baggage retrieving video to acquire the information aboutthe subject person who retrieves the baggage.

According to an embodiment of the present disclosure, thebaggage-scanned-image acquiring module includes: a second time-matchingunit configured to match a calculated time when the scanned image of thebaggage is generated with an actual time when the scanned image isgenerated, to acquire the scanned image of the baggage.

According to an embodiment of the present disclosure, the videoanalyzing module includes: a frame extracting unit configured to decodea stream of a video to extract a single frame; a marking unit configuredto mark an interested region and an interested line on an image of theframe; a detecting unit configured to detect a dynamic object in theinterested region; an open-close processing unit configured to acquire aforeground view and perform opening and closing process on theforeground view; a key-frame determining unit configured to, once in theprocessed foreground view, the dynamic object has crossed the interestedline, compare the current frame with a state of the preceding frame anddetermine whether the current frame is a key frame; and abaggage-information extracting unit configured to, if the current frameis a key frame, extract and store information about a baggage of thedynamic object in the key frame.

According to an embodiment of the present disclosure, the videoanalyzing module includes: a time-segment dividing unit configured todivide a real-time stream of video data into data of a plurality of timesegments; and a processing unit configured to process the data of aplurality of time segments in parallel.

According to an embodiment of the present disclosure, the system furtherincludes a storage module configured to store the information about thebaggage, the scanned image of the baggage and the information about thesubject person who retrieves the baggage in a system server.

According to an embodiment of the present disclosure, the informationabout the baggage includes at least one of: an image, a color and asize.

According to an embodiment of the present disclosure, the informationabout the subject person who retrieves the baggage can include at leastone of: an image of the face, a fragment of the baggage retrievingvideo.

According to an embodiment of the present disclosure, the scanned imagecan be an X-ray image.

According to yet another aspect of the present disclosure, a device forsecurity inspection, including a camera and a security inspectionmachine, the camera is disposed at an entrance side and/or an exit sideof the security inspection machine and configured to acquire a video ofa baggage and a video of a subject person before the baggage enters thesecurity inspection machine and/or after the baggage leaves the securityinspection machine; and the security inspection machine is configured toscan a baggage while the baggage is inside the security inspectionmachine to be scanned, to generate a scanned image of the baggage.

According to an embodiment of the present disclosure, the cameraincludes a first camera and a second camera, wherein the first camera isdisposed at the entrance side of the security inspection machine,configured to acquire an entrance baggage video before the baggageenters the security inspection machine, and the second camera isdisposed at the exit side of the security inspection machine, configuredto acquire and extract a video of the subject person who retrieves thebaggage after the baggage leaves the security inspection machine.

According to an embodiment of the present disclosure, the device furtherincludes a controlling unit, the controlling unit includes the system ofany of the above described.

According to the method, the system and the device for securityinspection of the present disclosure, by correlating and binding imageand information of the subject person, the baggage carried with him orher and the X-ray image of the baggage, images of the person, thebaggage carried with him or her and the scanned image of the baggage canbe correlated such that a person can correspond to his or her baggage.In this way, it can significantly improve the efficiency of securityinspection of the subject person and baggage carried with him or her.After a security accident happens, other relevant information can beautomatically searched out based on a package left on the spot orinformation about a suspect in a monitored video. Therefore, it canprovide correlation among information acquired in the securityinspection and improve the usability of information acquired in thesecurity inspection.

In addition, according to some embodiments, the method, the system andthe device for security inspection of the present disclosure can capturea baggage from a video and analyze properties of the baggage.

According to another embodiment, the method, the system and the devicefor security inspection of the present disclosure can improve performingefficiency of the video analyzing algorithm, increase the speed ofrecognizing a baggage and a human face, and improve the recognitionaccuracy.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary only and are notrestrictive of the present disclosure, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present disclosurewill become apparent from exemplary embodiments thereof described indetail with reference to accompanying drawings.

FIG. 1 is a flowchart illustrating a method for security inspectionaccording to an exemplary embodiment of the present disclosure;

FIG. 2 is a flowchart illustrating a method for analyzing a video basedon the method for security inspection of FIG. 1;

FIG. 3 is a flowchart illustrating a process for analyzing a video basedon the method for security inspection of FIG. 1;

FIG. 4 is a block diagram illustrating a system for security inspectionaccording to an exemplary embodiment of the present disclosure;

FIG. 5 is a block diagram illustrating a video analyzing module based onthe system for security inspection of FIG. 4; and

FIG. 6 is a schematic diagram illustrating a device for securityinspection according to an exemplary embodiment of the presentdisclosure.

DETAILED DESCRIPTION

Hereinafter, the exemplary embodiments will be described more fully withreference to the accompanying drawings. However, the exemplaryembodiments can be implemented in various manners, and should not beunderstood as limited to the embodiments set forth herein. Instead,these embodiments are provided to make the present disclosure morethorough and complete, and to fully convey the concept of the exemplaryembodiments to those skilled in the art. Throughout the accompanyingdrawings, like symbols represent like or the same structure, and thusthe redundant description will be omitted.

In addition, the features, structure or characteristics described can becombined in one or more embodiments in any suitable way. In thefollowing description, more specific details are provided to enablethorough understanding of the embodiments of the present disclosure.However, it should be appreciated by those skilled in the art that thetechnical solution of the present disclosure can be practiced withoutone or more of the particular details or can be practiced with othermethods, components, materials, devices or steps and so on. In somecases, known structure, methods, devices, implementation, material oroperation will not be illustrated in detail to avoid obscuration of thepresent disclosure.

In the accompanying drawings, a block diagram only illustratesfunctional modules each of which does not necessarily correspond to aseparate physical component one by one. That is, these functionalmodules can be implemented in software, or the whole or part of thesefunctional modules can be implemented in one or more hardware modules,for example, executed through software. Alternatively, these functionalmodules can also be implemented in various network and/or processordevices and/or micro controllers.

FIG. 1 is a flowchart illustrating a method for security inspectionaccording to an exemplary embodiment of the present disclosure.

As shown in FIG. 1, at step S110, before a baggage enters a securityinspection machine and/or after the baggage leaves the securityinspection machine, information about the baggage and information abouta subject person are acquired.

In an exemplary embodiment, before a baggage enters a securityinspection machine and/or after the baggage leaves the securityinspection machine, acquiring information about the baggage andinformation about a subject person can include: before the baggageenters a security inspection machine, acquiring the information aboutthe baggage and the information about the subject person by a cameradisposed at an entrance side of the security inspection machine. Theinformation about the baggage and the information about the subjectperson can be acquired at the same time by the same camera. In thiscase, through image processing, the information about the baggage andthe information about the subject person can be extracted separately.Alternatively, two cameras can be disposed at the entrance side of thesecurity inspection machine. In this case, one of the cameras isconfigured to acquire the information about the subject person, and theother of the cameras is configured to acquire the information about thebaggage.

In an exemplary embodiment, before a baggage enters a securityinspection machine and/or after the baggage leaves the securityinspection machine, acquiring information about the baggage andinformation about a subject person can include: after the baggage leavesthe security inspection machine, capturing a video about the baggagewhen the baggage is being retrieved by the subject person (hereinafterreferred to as baggage retrieving video), by a camera disposed at anexit side of the security inspection machine, and analyzing the baggageretrieving video to acquire the information about the baggage and theinformation about the subject person.

In an exemplary embodiment, before a baggage enters a securityinspection machine and/or after the baggage leaves the securityinspection machine, acquiring information about the baggage andinformation about a subject person can include: before the baggageenters the security inspection machine, acquiring information about thebaggage at an entrance side of the security inspection machine(hereinafter referred to as entrance baggage information); after thebaggage leaves the security inspection machine, acquiring a baggageretrieving video, and analyzing the baggage retrieving video to acquirethe information about the baggage at an exit side of the securityinspection machine (hereinafter referred to as exit baggage information)and the information about the subject person; and matching the entrancebaggage information with the exit baggage information through analgorithm (for example, time and/or properties of the baggage), toacquire the information about the baggage.

In an exemplary embodiment, before a baggage enters a securityinspection machine and/or after the baggage leaves the securityinspection machine, acquiring information about the baggage andinformation about a subject person can include: before the baggageenters the security inspection machine, acquiring the entrance baggageinformation and the information about the subject person at the entranceside of the security inspection machine (hereinafter referred to asentrance person information), by a camera disposed at the entrance sideof the security inspection machine; after the baggage leaves thesecurity inspection machine, acquiring a baggage retrieving video, andanalyzing the baggage retrieving video to acquire the exit baggageinformation and the information about the subject person at an exit sideof the security inspection machine (hereinafter referred to as exitperson information); and through an algorithm, matching the entrancebaggage information with the exit baggage information to acquire theinformation about the baggage, and matching the entrance personinformation with the exit person information to acquire the informationabout the subject person.

In an exemplary embodiment, acquiring information about the baggage caninclude: acquiring a video of the baggage; and analyzing the video ofthe baggage to acquire entrance baggage information about the baggage.

In an exemplary embodiment, acquiring information about a subject personcan include: acquiring a baggage retrieving video; and analyzing thebaggage retrieving video to acquire the information about the subjectperson who retrieves the baggage.

At step S120, while the baggage is inside the security inspectionmachine to be scanned, a scanned image of the baggage is acquired.

At step S130, the information about the baggage, the scanned image ofthe baggage and the information about the subject person are correlatedin a storage system. In an embodiment, acquiring the information aboutthe baggage and acquiring the information about the subject person caninclude analyzing a video.

In an exemplary embodiment, analyzing the baggage retrieving video toacquire the information about the subject person can include: analyzingthe baggage retrieving video to acquire the exit baggage information atthe exit side of the security inspection machine and the informationabout the subject person; and matching the exit baggage information withthe entrance baggage information, so as to identify the baggage when thebaggage leaves the security inspection machine.

In an exemplary embodiment, analyzing the baggage retrieving video toacquire the information about the subject person can include: matching atime when the baggage enters the security inspection machine with a timewhen the baggage leaves the security inspection machine, so as toidentify the baggage when the baggage leaves the security machine; andanalyzing the baggage retrieving video to acquire the information aboutthe subject person who retrieves the baggage.

In an exemplary embodiment, acquiring a scanned image of the baggage caninclude: matching a calculated time when the scanned image of thebaggage is generated with an actual time when the scanned image isgenerated, to acquire the scanned image of the baggage.

In an exemplary embodiment, the method can also include: storing theinformation about the baggage, the scanned image of the baggage and theinformation about the subject person who retrieves the baggage in asystem server.

In an exemplary embodiment, the information about the baggage caninclude at least one of the following: an image, a color and a size.

In an exemplary embodiment, the information about the subject person whoretrieves the baggage can include at least one of the following: animage of the face, a fragment of the baggage retrieving video. The imageof the face can be acquired by utilizing any one of an existing faceprocessing algorithm, which is not limited in the present disclosure.

In an exemplary embodiment, the scanned image can be an X-ray image.

In an exemplary embodiment, a subject person (such as a subway orrailway passenger) puts his or her packed baggage (briefly referred toas baggage) on a conveyor belt of a security inspection machine. At thistime, a first camera disposed at an entrance side of the securityinspection machine acquires a video of the baggage. A system serveracquires the video of the baggage, analyzes the video of the baggageusing a video analyzing algorithm to obtain information about thebaggage. The information about the baggage can include properties of thebaggage, such as a color, a size, a brand, wear and tear and evenmaterial of the baggage, and so on. Afterwards, the baggage enters thesecurity inspection machine with the conveyor belt of the securityinspection machine at a uniform speed. Then, the security inspectionmachine scans the baggage to generate a scanned image of the baggage,such as an X-ray image. The system server acquires the X-ray image ofthe baggage. Next, the baggage keeps moving forward with the conveyorbelt of the security inspection machine, and comes out from an exit sideof the security inspection machine. After the security inspection of thebaggage is completed, the subject person retrieves his or her baggagefrom the conveyor belt of the security inspection machine. At this time,a second camera installed at the exit side of the security inspectionmachine captures a video of the baggage and a video of the subjectperson corresponding to the baggage, for example, a video of the face ofthe subject person. The system server acquires the video, and extractsthe face image of the subject person through a face identifyingalgorithm. The system server binds and correlates the information aboutthe baggage (including the entrance baggage information, the exitbaggage information and information about matching the both), the faceimage of the subject person and the X-ray image of the baggage.

In an exemplary embodiment, the security officer can retrieve othercorrelated information based on a property of a face or a property of abaggage (that is, searching for a baggage based on a person or searchingfor a person based on a baggage). If the security officer finds asuspect person, the security officer can search based on the face of thesuspect person. If correlated information is searched out, a video ofthe baggage, an image of the baggage and an X-ray image of the baggageof the suspect person on the spot can be acquired. In another example,if the security officer finds a suspect baggage, the security officercan search a suspect person based on the property of the baggage. Ifcorrelated information is searched out, information about the face ofthe suspect person and an X-ray image of the baggage can be acquired.

FIG. 2 is a flowchart illustrating a method for analyzing a video basedon the method for security inspection of FIG. 1.

As shown in FIG. 2, at step S210, a stream of a video is decoded toextract a single frame.

At step S220, an interested region and an interested line are marked onan image of the frame.

In an exemplary embodiment, the video of the baggage is analyzed, and aninput stream of the video is decoded to extract a single frame as acurrent frame. An interested region is marked on the image of thecurrent frame according to relevance to the baggage. For example, whenthe baggage is put down, the camera will acquire an image of a hand ofthe subject person and an image of the baggage at the same time. Inanother example, when the baggage moves forward with the conveyor beltof the security inspection machine to a lead curtain of the securityinspection machine, the lead curtain flips up, causing the camera tocapture an image of the lead curtain and an image of the baggage at thesame time. In this case, in the image of the frame, the interestedregion will be a region excluding the portion of the hand and the leadcurtain, and only including the baggage.

At step S230, an object with position changed through the frames ofimages (hereinafter referred to as a dynamic object) in the interestedregion is detected.

At step S240, a foreground view is acquired and is processed with anopen-close algorithm.

In an exemplary embodiment, the term foreground view can be understoodwith respect to a background view and a further-background view. Theforeground view, background view and the further-background view overlapwith one another. The further-background view is the lowest, thebackground view is located over the further-background view, and theforeground is located on the uppermost surface.

In an exemplary embodiment, the open-close algorithm is a basicoperation utilizing morphology, used for observing and processing animage, and improving the quality of the image; and describing anddefining geometric parameters and characteristic of the image, such asan area, a perimeter, connectivity, a particle size, skeleton andorientation.

In an exemplary embodiment, morphology operation for a binary image ormorphology operation for a gray scale image can also be used. Basicmorphology operations for a binary image include erosion and dilation.Erosion is a process to eliminate all boundary points of an object. Theresult of the process is to cause an area of remaining part of theobject smaller than the original object by a size of a pixel along theperiphery. If the object is a circle, its diameter will be reduced bytwo pixels after one time of erosion. If at a certain point the objectthere are less than three pixels communicated with each other in anydirection, the object will split into two objects at that point afterone time of erosion. Dilation is a process to incorporate all points inthe background in contact with an object into the object. The result ofthe process is to increase the area of the object by correspondingpoints. If the object is a circle, its diameter will be increased by twopixels after one time of dilation. If two objects are separated by lessthan three pixels at a certain point in any direction, the two objectswill be communicated at this point.

Erosion can be used for eliminating small noise regions in an image, anddilation can be used for filling holes in an object. Operation offirstly performing erosion on an image and then performing dilation onthe image is referred to as opening operation. It can eliminate tinyobjects, isolating objects at a tiny point, and smoothening boundary ofa large object without significantly changing its area. On the otherhand, operation of firstly performing dilation on an image and thenperforming erosion on the image is referred to as closing operation. Itcan be used for filling tiny holes inside an object, connecting adjacentobjects, and smoothening boundary of an object without significantlychanging its area.

In general, after an image with noise is transformed to a binary imagewith respect to a threshold, the boundary is not smooth, the region ofthe object has some erroneous holes and the background region has somesmall noise objects scattered therein. Continuous opening operation andclosing operation can significantly improve these defects. After severaltimes of iterative erosion and the same times of dilation, a desirableeffect can be achieved. After such processing, the noise points in theimage can be desirably eliminated, and the edge of the image can besmoothened.

At step S250, it is determined whether the dynamic object has crossedthe interested line in the processed frame. If the moving object hascrossed the interested line, the process of the method proceeds to thenext step; otherwise, the process returns to step S230.

At step S260, the current frame is compared with a state of a precedingframe.

At step S270, it is determined whether the current frame is a key framebased on the comparison result. If the current frame is a key frame, theprocess of the method proceeds to the next step; otherwise, the processreturns to step S210 to extract the next frame.

At step S280, information about a baggage of the dynamic object in thekey frame is extracted and stored.

In an exemplary embodiment, the method can also include: marking aninterested line on the image of the frame; detecting a dynamic object inthe interested region; acquiring a foreground view and performingopening and closing process on the foreground view; once it is detectedthat the dynamic object has crossed the interested line in the processedframe, comparing the current frame with a state of a previously storedpreceding frame. Here, the state of the preceding frame can be, forexample, whether the dynamic object has crossed the interested line inthe preceding frame. If the state of the preceding frame is that thedynamic object has crossed the interested line in the preceding frame,it means that the information about the baggage has been stored. In thiscase, the operation of storing and extracting the information of thebaggage may be skipped for the current frame. Otherwise, if the state ofthe preceding frame is that the dynamic object has not crossed theinterested line in the preceding frame, it means that information of thebaggage has not be stored and the current frame is the first frame inwhich the dynamic object has crossed the interested line. In this case,the current frame is stored as the key frame, and the information aboutthe baggage of the dynamic object in the key frame is extracted. In thisway, it can reduce the storage space of the system, improve theprocessing speed of the algorithm, and eliminate redundant storage andextraction of the same baggage.

FIG. 3 is a flowchart illustrating a process for analyzing a video basedon the method for security inspection of FIG. 1.

As shown in FIG. 3, at step S310, a real-time stream of video data isdivided into data of a plurality of time segments.

At step S320, the data of a plurality of time segments are processed inparallel.

In an exemplary embodiment, in order to improve the performance of thealgorithm, data structure is built by utilizing big data technology, anddeployed on an Apache Spark architecture. By utilizing Spark streamingtechnology, it can improve performing efficiency of the algorithm,increase the speed of recognizing a baggage and a human face, andimprove the recognition accuracy. Moreover, a massive of videos andimages can be stored. Spark Streaming is a computational framework of areal-time stream built on Spark. Spark Streaming expands the capabilityof Spark in handling massive stream data. The basic principle of SparkStreaming is dividing input stream of data into time segments (thesecond level), and processing data of each time segment in a mannersimilar to parallel batch processing. Here, a video stream can be areal-time stream for input. The Spark Streaming technology can be usedfor loading a video processing algorithm to achieve highly efficientalgorithm performance. However, other technology for processing imagesin parallel can also be used, which is not limited herein.

With the method for security inspection according to the embodiments ofthe present disclosure, it can utilize the highly efficient performanceof video analyzing algorithm provided by the big data technology. It cansignificantly increase the speed of recognizing a baggage and a humanface, and improve the recognition accuracy.

FIG. 4 is a block diagram illustrating a system for security inspectionaccording to an exemplary embodiment of the present disclosure.

As shown in FIG. 4, the system 400 includes: a baggage-informationacquiring module 410 configured to, before a baggage enters a securityinspection machine and/or after the baggage leaves the securityinspection machine, acquire information about the baggage andinformation about a subject person; a baggage-scanned-image acquiringmodule 420 configured to, while the baggage is inside the securityinspection machine to be scanned, acquire a scanned image of thebaggage; a person-information acquiring module 430 configured to, beforea baggage enters a security inspection machine and/or after the baggageleaves the security inspection machine, acquire information about asubject person corresponding to the baggage; a correlation module 450configured to correlate the information about the baggage, the scannedimage of the baggage and the information about the subject person in astorage system; and a video analyzing module 440 configured to analyze avideo for the baggage-information acquiring module 410 and theperson-information acquiring module 430.

In an exemplary embodiment, the baggage-information acquiring module 410includes: a baggage-video acquiring unit configured to acquire a videoof the baggage; and a baggage analyzing unit configured to analyze thevideo of the baggage to acquire entrance baggage information about thebaggage.

In an exemplary embodiment, the person-information acquiring module 430includes: a baggage-retrieving-video acquiring unit configured toacquire a baggage retrieving video; and a baggage-retrieving analyzingunit configured to analyze the baggage retrieving video to acquire theinformation about the subject person.

In an exemplary embodiment, the baggage-retrieving analyzing unitincludes: a baggage analyzing sub-unit configured to analyze the baggageretrieving video to acquire an exit baggage information; a retrieveranalyzing sub-unit configured to analyze the baggage retrieving video toacquire the information about the subject person who retrieves thebaggage; and a baggage matching sub-unit configured to match theentrance baggage information with the exit baggage information, so as toidentify the baggage when the baggage leaves the security inspectionmachine.

In an exemplary embodiment, the baggage-retrieving analyzing unitincludes: a first time-matching sub-unit configured to match a time whenthe baggage enters the security inspection machine with a time when thebaggage leaves the security inspection machine, so as to identify thebaggage when the baggage leaves the security machine; and a retrieveranalyzing sub-unit configured to analyze the baggage retrieving video toacquire the information about the subject person who retrieves thebaggage.

In an exemplary embodiment, the baggage-scanned-image acquiring module420 includes: a second time-matching unit configured to match acalculated time when the scanned image of the baggage is generated withan actual time when the scanned image is generated, to acquire thescanned image of the baggage.

In an exemplary embodiment, the video analyzing module 440 includes: atime-segment dividing unit configured to divide a real-time stream ofvideo data into data of a plurality of time segments; and a processingunit configured to process the data of a plurality of time segments arein parallel.

In an exemplary embodiment, the system also includes a storage moduleconfigured to store the information about the baggage, the scanned imageof the baggage and the information about the subject person whoretrieves the baggage in a system server.

In an exemplary embodiment, the information about the baggage caninclude at least one of the following: an image, a color and a size.

In an exemplary embodiment, the information about the subject person whoretrieves the baggage can include at least one of the following: animage of the face, a fragment of the baggage retrieving video.

In an exemplary embodiment, the scanned image can be an X-ray image.

Specific implementation of each of the modules of the system in theabove embodiment have been described in detail with reference torelevant method embodiments, therefore a detailed description of themodules will not be elaborated herein.

FIG. 5 is a block diagram illustrating the video analyzing module 440based on the system for security inspection of FIG. 4.

As shown in FIG. 5, the video analyzing module 440 includes: a frameextracting unit 510 configured to decode a stream of a video to extracta single frame; a marking unit 520 configured to mark an interestedregion and an interested line on an image of the frame; a detecting unit530 configured to detect a dynamic object in the interested region; anopen-close processing unit 540 configured to acquire a foreground viewand perform opening and closing process on the foreground view; akey-frame determining unit 550 configured to determine whether a frameis a key frame by comparing the frame with a preceding frame to seewhether the dynamic object has crossed the interested line after thedynamic object leaves the security inspection machine; and abaggage-information extracting unit 560 configured to extract and storethe information about the baggage of the dynamic object in the keyframe.

FIG. 6 is a schematic diagram illustrating a device 600 for securityinspection according to an exemplary embodiment of the presentdisclosure.

As shown in FIG. 6, the device 600 for security inspection includescameras 630 and 640 and a security inspection machine 610. The cameras630 and 640 are disposed at an entrance side and/or an exit side of thesecurity inspection machine 610, and configured to acquire a video of abaggage and a video of a subject person before the baggage enters thesecurity inspection machine 610 and/or after the baggage leaves thesecurity inspection machine 610. The security inspection machine 610 isconfigured to scan a baggage while the baggage is inside the securityinspection machine to be scanned, so as to generate a scanned image ofthe baggage.

In an exemplary embodiment, the camera 630 can be disposed at theentrance side of the security inspection machine. Before the baggageenters the security inspection machine, the camera can acquire theinformation about the baggage and the information about the subjectperson. The information about the baggage and the information about thesubject person can be acquired at the same time by the same camera. Inthis case, through image processing, the information about the baggageand the information about the subject person can be extractedseparately. Alternatively, two cameras can be disposed at the entranceside of the security inspection machine. In this case, one of thecameras is configured to acquire the information about the subjectperson, and the other of the cameras is configured to acquire theinformation about the baggage.

In an exemplary embodiment, the camera 640 can be disposed at the exitside of the security inspection machine. After the baggage leaves thesecurity inspection machine, the camera at the security inspectionmachine can acquire a baggage retrieving video. By analyzing the baggageretrieving video, the information about the baggage and the informationabout the subject person can be acquired.

In an exemplary embodiment, the cameras include a first camera 630 and asecond camera 640. The first camera 630 is disposed at the entrance sideof the security inspection machine 610, configured to acquire anentrance-baggage video before the baggage enters the security inspectionmachine 610. The second camera 640 is disposed at the exit side of thesecurity inspection machine 610, configured to acquire the baggageretrieving video after the baggage leaves the security inspectionmachine 610. By analyzing the baggage retrieving video, exit baggageinformation and the information about the subject person can beacquired. Through an algorithm (for example, time and/or properties ofthe baggage), the entrance baggage information is matched with the exitbaggage information to acquire the information about the baggage.

In an exemplary embodiment, before the baggage enters the securityinspection machine 610, entrance baggage information and entrance personinformation are acquired by the first camera 630 disposed at theentrance side of the security inspection machine. After the baggageleaves the security inspection machine 610, a baggage retrieving videois acquired by the second camera 640 disposed at the exit side of thesecurity inspection machine. By analyzing the baggage retrieving video,exit baggage information and exit person information can be acquired.Through an algorithm, the entrance baggage information is matched withthe exit baggage information to acquire the information about thebaggage, and the entrance person information is matched with the exitperson information to acquire the information about the subject person.

In an exemplary embodiment, the example of the first camera 630 and thesecond camera 640 does not limit the number of the cameras. The firstcamera 630 can be one camera or more than one camera. Similarly, thesecond camera 640 can also be one camera or more than one camera.Although in FIG. 6, the cameras are installed at an upper side of thesecurity inspection machine 610, in practice, the cameras can beinstalled at any position at the entrance side and/or exit side of thesecurity inspection machine 610, and even can be disposed at theentrance of the door of a subway or railway station, as long as thecamera can capture images about humans and/or objects within itsmonitoring region. This is not limited in the present disclosure.

In an exemplary embodiment, the security inspection machine 610 canfurther include an X-ray machine with a head portion 620, a lead curtain650 and a conveyor belt 660.

In an exemplary embodiment, the baggage retrieving video contains thebaggage 670 and the subject person 680.

In an exemplary embodiment, the subject person 680 puts the baggage 670on the conveyor belt 660 of the security inspection machine 610. Thefirst camera 630 acquires a video of the baggage 670. An image of thevideo of the baggage 670 is analyzed through a video analyzing algorithmto acquire properties of the baggage. When the baggage 670 moves on theconveyor belt 660 to the lead curtain 650 at the entrance side of thesecurity inspection machine 610, this position of the baggage can bedetected through the video analyzing algorithm and a time instance t1 isrecorded. According to a horizontal distance s between the lead curtain650 of the security inspection machine 610 and the head portion 620 ofthe X-ray machine of the security inspection machine 610, and a movingspeed v of the conveyor belt 660 of the security inspection machine 610,a time t2 when the X-ray image of the baggage 670 is generated can becalculated as (t2=t1+s/v). Therefore, the X-ray image at the timeinstance t2 shows the baggage. At the exit side of the securityinspection machine 610, the image of the baggage 670 and the face of thesubject person 680 can be detected through the video analyzingalgorithm.

In an exemplary embodiment, by matching with the images of the baggageat the entrance and the exit of the security inspection machine throughthe video analyzing algorithm, the face image of the subject person 680can be extracted.

In an exemplary embodiment, according to a horizontal distance S betweenthe lead curtain 650 at the entrance side of the security inspectionmachine 610 and the head portion 620 of the X-ray machine of thesecurity inspection machine 610, and a moving speed v of the conveyorbelt 660 of the security inspection machine 610, a time t3 when thebaggage 670 comes out from the exit side of the security inspectionmachine 610 can be calculated as (t3=t1+S/v). In this way, the baggageat the entrance can be matched with the baggage at the exit. The imageof the baggage at the entrance, the X-ray image of the baggage at thetime instance t2 and the face image of the subject person 680 at theexit can be correlated or bound with one another.

In an exemplary embodiment, the time instance t3 when the baggage 670reaches the lead curtain 650 at the exit side of the security inspectionmachine 610 can be recorded. According to a horizontal distance S1between the lead curtain 650 at the exit side of the security inspectionmachine 610 and the head portion 620 of the X-ray machine of thesecurity inspection machine 610, and a moving speed v of the conveyorbelt 660 of the security inspection machine 610, a time t2 when thebaggage 670 reaches the head portion 620 of the X-ray machine of thesecurity inspection machine 610 can be calculated as (t2=t3−S1/v). Theimage of the baggage acquired at the exit, the X-ray image of thebaggage at the time instance t2 and the face image of the subject person680 at the exit can be correlated or bound with one another.

In an exemplary embodiment, the device further includes a controllingunit 690, the controlling unit is capable of performing the method forsecurity inspection as described in the above embodiments.

FIG. 1 is a flowchart illustrating a method for security inspectionaccording to an exemplary embodiment of the present disclosure. Themethod can utilize for example the system and the device for securityinspection as shown in FIG. 4, 5 or 6, to which the present disclosureis not limited. It should be noted that, FIGS. 1, 2 and 3 are merely forillustration of the process of the method according to the embodimentsof the present disclosure, rather than for limitation purpose. It shouldbe readily understood that FIGS. 1, 2 and 3 do not define or limit thetime sequence of the processing. In addition, it should also be readilyunderstood that these steps can be performed in a plurality ofmodules/processes/threads in parallel or not in parallel, for example.

From the above description of the embodiments, it should be readilyunderstood by those skilled in the art that the exemplary embodimentsdescribed herein can be implemented in software, or can be implementedby combination of software with necessary hardware. Therefore, thetechnical solution of the embodiments of the present disclosure can beembodied in software product which is stored on a non-transient storagemedium (can be a CD-ROM, a U disk, a movable hard disk and so on) orover network, and can include instructions to cause a computer (can be apersonal computer, a server, a mobile terminal, or a network device andso on) to perform the method according to the embodiments of the presentdisclosure.

It should be understood by those skilled in the art that theaccompanying drawings are only illustration of the exemplaryembodiments, the modules or steps in the accompanying drawings may benot essential for the present disclosure. Therefore, they do notconstitute limitation to the protective scope of the present disclosure.

It should be understood by those skilled in the art that the abovemodules can be distributed in devices according to the description ofthe embodiments, or can be located in one or more devices by modifyingthe embodiments of the present disclosure. The modules of theembodiments can be combined into one module, or can be further dividedinto more than one sub-modules.

The exemplary embodiments of the present disclosure have beenillustrated and described above. It should be understood that, thepresent disclosure is not limited to the embodiments disclosed. Instead,the present disclosure intends to cover all the alteration andequivalent replacement within the spirit and scope of the appendingclaims.

What is claimed is:
 1. A method for security inspection, comprising:before a baggage enters a security inspection machine and after thebaggage leaves the security inspection machine, acquiring informationabout the baggage and information about a subject person; while thebaggage is inside the security inspection machine to be scanned,acquiring a scanned image of the baggage; and correlating theinformation about the baggage, the scanned image of the baggage and theinformation about the subject person in a storage system, wherein theacquiring the information about the baggage and the information aboutthe subject person comprises analyzing a video, wherein the acquiringthe information about the baggage comprises: acquiring a video of thebaggage; and analyzing the video of the baggage to acquire entrancebaggage information about the baggage, wherein the acquiring theinformation about the subject person comprises: acquiring a baggageretrieving video; and analyzing the baggage retrieving video to acquirethe information about the subject person who retrieves the baggage, andwherein the analyzing the baggage retrieving video to acquire theinformation about the subject person comprises: matching a time when thebaggage enters the security inspection machine with a time when thebaggage leaves the security inspection machine to identify the baggagewhen the baggage leaves the security machine; and analyzing the baggageretrieving video to acquire the information about the subject person whoretrieves the baggage.
 2. The method of claim 1, wherein acquiring ascanned image of the baggage comprises: matching a calculated time whenthe scanned image of the baggage is generated with an actual time whenthe scanned image is generated, to acquire the scanned image of thebaggage.
 3. The method of claim 1, wherein analyzing a video comprises:decoding a stream of a video to extract a single frame; marking aninterested region and an interested line on an image of the frame;detecting a dynamic object in the interested region; acquiring aforeground view and performing opening and closing process on theforeground view; once in the processed foreground view, the dynamicobject has crossed the interested line, comparing the current frame witha state of the preceding frame and determining whether the current frameis a key frame; if the current frame is a key frame, extracting andstoring infor aation about a baggage of the dynamic object in the keyframe; and if it is not a key frame, extracting a next frame.
 4. Themethod of claim 1, wherein analyzing a video comprises: dividing areal-time stream of video data into data of a plurality of timesegments; and processing the data of a plurality of time segments inparallel.
 5. The method of claim 1, wherein the method further comprisesstoring the information about the baggage, the scanned image of thebaggage and the information about the subject person who retrieves thebaggage in a system server.
 6. A device for security inspection,comprising a camera and a security inspection machine, wherein, thecamera is disposed at an entrance side and an exit side of the securityinspection machine and configured to acquire a video of a baggage and avideo of a subject person before the baggage enters the securityinspection machine and/or after the baggage leaves the securityinspection machine; and the security inspection machine is configured toscan a baggage while the baggage is inside the security inspectionmachine to be scanned, to generate a scanned image of the baggage,wherein the camera comprises a first camera and a second camera, whereinthe first camera is disposed at the entrance side of the securityinspection machine configured to acquire an entrance baggage videobefore the baggage enters the security inspection machine, wherein thesecond camera is disposed at the exit side of the security inspectionmachine, configured to acquire and extract a video of the subject personwho retrieves the baggage after the baggage leaves the securityinspection machine, and wherein the device further comprises acontrolling unit configured to perform the method of claim
 1. 7. Thedevice of claim 6, wherein acquiring a scanned image of the baggagecomprises: matching a calculated time when the scanned image of thebaggage is generated with an actual time when the scanned image isgenerated, to acquire the scanned image of the baggage.
 8. The device ofclaim 6, wherein analyzing a video comprises: decoding a stream of avideo to extract a single frame; marking an interested region and aninterested line on an image of the frame; detecting a dynamic object inthe interested region; acquiring a foreground view and performingopening and closing process on the foreground view; once in theprocessed foreground view, the dynamic object has crossed the interestedline, comparing the current frame with a state of the preceding frameand determining whether the current frame is a key frame; if the currentframe is a key frame, extracting and storing information about a baggageof the dynamic object in the key frame; and if it is not a key frame,extracting a next frame.
 9. The device of claim 6, wherein analyzing avideo comprises: dividing a real-time stream of video data into data ofa plurality of time segments; and processing the data of a plurality oftime segments in parallel.
 10. The device of claim 6, wherein the methodfurther comprises storing the information about the baggage, the scannedimage of the baggage and the information about the subject person whoretrieves the baggage in a system server.